Markov Chain and Hidden Markov Model

Figure 1. Finite state machine for a Markov chain X0 → X1 → X2 → · · · → Xn where the random variables Xi’s take values from I = {S1, S2, S3}. The numbers T (i, j)’s on the arrows are the transition probabilities such that Tij = P (Xt+1 = Sj|Xt = Si). Definition 1.2. We say that (Xn)n≥0 is a Markov chain with initial distribution λ and transition matrix T if (i) X0 has distribution λ; (ii) for n ≥ 0, conditional on Xn = i, Xn+1 has distribution (Tij : j ∈ I) and is independent of X0, . . . , Xn−1. By the Markov property we have P (X0, . . . , Xn) = P (X0)P (X1|X0) · · ·P (Xn|X0, . . . , Xn−1) (1)